Code image distribution in a multi-node network of processors

ABSTRACT

A multi-node processing network has a plurality of processors coupled in the network. The processors have a minimally operational state, e.g., upon being rebooted, and have a fully operational state employing a code image. The processors, when in the minimally operational state, request the code image from the network. The processors have a non-volatile memory storing code for the minimally operational state, which is sufficient to at least provide the code image request. A master source is coupled in the network, the master source having at least a code image for broadcasting on the network. The master source, upon receiving a code image request, waits a predetermined time period, the predetermined time period allowing any additional processor to reach the minimally operational state. Upon completion of the predetermined time period, the master source broadcasts the code image on the network. The processor may store the code image in volatile memory since it may easily be requested.

FIELD OF THE INVENTION

[0001] This invention relates to multi-node networks of embedded systemsemploying processors, each processor operating in accordance with a codeimage, and, more particularly, to insuring that each processor of thenetwork has its code image, for example, when the processor or thenetwork is powered on.

BACKGROUND OF THE INVENTION

[0002] Typically, each processor of a multi-node network of embeddedsystems employing processors has a high speed RAM (Random Access Memory)for storing a code image which, when executed, operates the processor.An example of an embedded system having a plurality of modules withprocessors at nodes of the system, comprises an automated data storagelibrary, which stores removable data storage media in storage shelves,which has a plurality of data storage drives to read and/or write dataon the data storage media, and which has at least one robot to transportthe data storage media between the storage shelves and the data storagedrives, with processors at the modules to operate the library, and witha network to couple the modules of the embedded library system. Such alibrary system may comprise a hundred or more data storage driveprocessors and tens of library processors. It can become a realchallenge to maintain consistent code levels in a network of a largenumber of nodes.

[0003] Some multi-node networks, such as LANs (Local Area Networks), areemployed for coupling together a number of like components, such as PCs(Personal Computers), with peripheral devices, such as printers. In atleast one instance, such as discussed by U.S. Pat. No. 5,815,722,executable code files for the specific devices may be updated bydownloading directly to RAM. In the patent, a communication programoperates to broadcast an inquiry on the LAN to a specific network board,to receive location information of the designated board, e.g., of aprinter. The executable file is then directly downloaded into RAM on thedesignated board through the LAN. However, to prevent loss of theexecutable code when any of the devices or boards is powered off, theRAM is non-volatile (called NVRAM).

[0004] The executable code for printers and similar devices isexceptionally small, requiring only a very small NVRAM). In more majorsystems, such as automated data storage libraries, a module processormay comprise a processor of the power of a workstation or PC, and theexecutable code for each module processor is quite large.

[0005] As a result, typically, each processor of a multi-node networkhas a PROM (Programmable Read-Only Memory) or a ROM (Read-Only Memory),which stores a power-on sequence, and which must store a copy of thecode image to prevent loss. A power-on sequence for any of the modulesof an embedded system involves the execution of power-on code that isstored in the PROM or ROM device, where a module may comprise anassembly, subassembly or a circuit board. The code may first test thecomponents of the module, and then may transfer the code image to thefaster RAM.

[0006] Thus, the need for both NVRAMs, PROMs or ROMs and a fast RAM isvery expensive. The requirement that a PROM or ROM have sufficientcapacity to store the code image for the associated RAM, or that a NVRAMbe used, adds significantly to the expense of each module. Whenmultiplied by a high number of modules, the expense can become asignificant factor in the cost of the full system. Further, should acode image be updated, failures of the update process are not uncommon,for example, if power is lost during a code update, and may even cripplethe node entirely.

SUMMARY OF THE INVENTION

[0007] An object of the present invention is to eliminate the need for alarge NVRAM, PROM or ROM.

[0008] Another object of the present invention is to provide code imagesto processors of a multi-node processing network, for example, when theprocessor or the network is powered on, without requiring a highcapacity NVRAM, PROM or ROM.

[0009] Disclosed are a multi-node processing network, method, andcomputer program product, with a plurality of processors coupled in thenetwork. The processors have a minimally operational state, e.g., uponbeing rebooted, and have a fully operational state employing a codeimage.

[0010] In accordance with the present invention, the processors, when inthe minimally operational state, request the code image from thenetwork. The processors may have a non-volatile memory storing code forthe minimally operational state which is sufficient to at least providethe code image request. A master source is coupled in the network, themaster source having at least a code image for broadcasting on thenetwork. The master source, upon receiving a code image request, waits apredetermined time period, the predetermined time period allowing anyadditional processor to reach the minimally operational state. Uponcompletion of the predetermined time period, the master sourcebroadcasts the code image on the network.

[0011] In another aspect of the present invention, the processors,additionally, upon the broadcast of the code image, receive andimplement the code image only if the processor is in the minimallyoperational state.

[0012] Thus, without requiring a high capacity NVRAM, PROM or ROM ateach processor, the present invention provides code images to processorsof a multi-node processing network, but does not interrupt a currentlyoperating processor.

[0013] A code image for the processors may be updated by first updatingthe master source, and rebooting all of the processors that are on thenetwork and, upon being rebooted, the processors will request the codeimage, and the master source will supply the updated code image. Inanother aspect of the present invention, in which the processors areprovided for modules of a redundant system having at least two sets ofredundant modules, at least one set of redundant modules is rebooted,such that the processors of the modules reach the minimally operationalstate and request the code image from the network. The redundant modulesmay comprise all or part of the nodes in the complete system, and maycomprise as little as a single node which is duplicated to be redundant.The master source, upon receiving a code image request, waits apredetermined time period, the predetermined time period allowing anyadditional processor to reach the minimally operational state. Themaster source, upon completion of the predetermined time period,broadcasts the code image on the network, such that the processorsrequesting the code image become fully operational. Then, a remainingset of redundant modules is rebooted, so that the processors of themodules reach the minimally operational state and request the code imagefrom the network. The master source, upon receiving a code imagerequest, waits a predetermined time period, the predetermined timeperiod allowing any additional processor to reach the minimallyoperational state, and, upon completion of the predetermined timeperiod, broadcasts the code image on the network, such that theprocessors of the remaining set of redundant modules requesting the codeimage become fully operational. In this manner, the code images for theentire redundant system have been updated without requiring that theoperation of the entire system be interrupted.

[0014] As the result, the present invention eliminates the need tomaintain and update code levels on individual processors.

[0015] For a fuller understanding of the present invention, referenceshould be made to the following detailed description taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016]FIG. 1 is a block diagram of a multi-node processing networkarranged in accordance with the present invention;

[0017]FIG. 2 is a diagrammatic representation of the code stored at amaster source of the network of FIG. 1;

[0018]FIG. 3 is an isometric view of an automated data storage libraryemploying the network of FIG. 1;

[0019]FIG. 4 is a block diagram of the automated data storage library ofFIG. 3;

[0020]FIG. 5 is a flow chart depicting computer implemented embodimentsof the method of the present invention for distributing code images in anetwork system; and

[0021]FIG. 6 is a flow chart depicting a computer implemented embodimentof a method for updating code images for a redundant system inaccordance with the present invention;

DETAILED DESCRIPTION OF THE INVENTION

[0022] This invention is described in preferred embodiments in thefollowing description with reference to the Figures, in which likenumbers represent the same or similar elements. While this invention isdescribed in terms of the best mode for achieving this invention'sobjectives, it will be appreciated by those skilled in the art thatvariations may be accomplished in view of these teachings withoutdeviating from the spirit or scope of the invention.

[0023]FIG. 1 illustrates a multi-node processing network system 100 witha plurality of processors 105 coupled in a network 106, together with amaster source 108. An example of a network comprises a multi-node systemof embedded processors. The embedded system comprises a plurality ofmodules with processors at nodes of the system, and the network servesto interconnect the modules 110 of the system, where the modules haveprogrammable processors 105 to operate the modules and their components111, thereby operating the system. In the illustrated example, theprocessors 105 each comprises a processor interface 112 coupling theprocessor in the network, a non-volatile memory 113, such as a ROM, forstoring code comprising at least a boot program, a processor memory 114,such as a high speed RAM, storing a code image providing a fullyoperational state of the processor, and a processing unit 115 coupled tothe non-volatile memory 113, the processor memory 114 and the processorinterface 112.

[0024] In accordance with the present invention, the processors 105 havea minimally operational state, e.g., upon being rebooted, employing theboot program code of the non-volatile memory 113, and have a fullyoperational state employing a code image stored in the processor memory114. Herein, “reboot” may comprise (a) a power-on of a processor, (b) areset of a processor, (c) a software instruction or command to cause aprocessor to run its boot code, or (d) any other initialization processresulting in running the boot code.

[0025] The processor memory 114 is typically a high speed RAM thatstores a code image which, when executed, operates the processor. Thecode image stored in the high speed RAM is lost whenever the processoris powered off. If a processor is powered off, the boot program storedin the non-volatile memory 113 becomes operational upon a power-on resetof the processor. As discussed above, the typical non-volatile memory ofthe prior art additionally contains the entire code image to preventloss of the executable code when any of the modules is powered off.

[0026] Further in accordance with the present invention, thenon-volatile memory 113 is of relatively small capacity, and onlycontains code for the minimally operational state which is sufficient toat least provide a code image request. As an example, many processorsmay include a flash memory 113 as an off-the-shelf product, making aseparate component unnecessary. Specifically, when a processor 105 is inthe minimally operational state, the processor employs the boot programto request the code image from the network 106. The master source 108 iscoupled in the network, and has at least the requested code image forbroadcasting on the network.

[0027] The master source 108 comprises a master interface 120 coupled inthe network, a master processor 125 coupled to the master interface, anda non-volatile memory 126 coupled to the master processor 125 whichstores at least the code image for the processors of the network. Themaster source may also comprise a functional node of the network with aRAM, module components, etc. In accordance with the present invention,upon receiving a code image request from a processor 105 at the masterinterface 120, the master source 108 waits a predetermined time period,the predetermined time period allowing any additional processor 105 toreach the minimally operational state, and, upon completion of thepredetermined time period, the master source 108 broadcasts the codeimage stored in the non-volatile memory 126, via the master interface120, on the network 106. The predetermined time period may be renewed orextended, as is discussed hereinafter.

[0028] Thus, without requiring a high capacity NVRAM, PROM or ROM ateach processor for storing the entire code image, the present inventionprovides code images to processors of a multi-node processing network.

[0029] In another aspect of the present invention, the processors 105,upon the broadcast of the code image by the master source, receive andimplement the code image only if the processor is in the minimallyoperational state. In this manner, no running fully operationalprocessor is interrupted or disrupted to interject a new code image.

[0030] Referring additionally to FIG. 2, the processors 105 of amulti-node network of embedded systems 100 are associated with modulesthat perform various functions in the system. Thus, the processors mayemploy correspondingly different code images. As the result, the mastersource 108 of the present invention may be required to store multiplecode images. An example of the code stored in non-volatile memory 126 ofmaster source 108 is depicted in FIG. 2, and comprises a code image 130for operating the master source 108 and other code images 131-133 foroperating the various processors 105. A code image may comprise acomputer program product usable with a programmable computer, thecomputer program product having computer readable program code embeddedtherein.

[0031] In the instance where the master source 108 also comprises afunctional node of the network, in one alternative, code image 130 maycomprise both the master source code and the functional node code. Inanother alternative, the code image 130 may comprise the master sourcecode, and one of the code images 131-133 may comprise the functionalnode code. A redundant master source 108 may also be provided and mayhave the same, a different, or no functional node code.

[0032] The master source may distribute the code images 131-133 eitherby 1) providing and broadcasting one code image for any code imagerequest, where the one master source code image comprises a combinationof the different code images 131-133, or by 2) selecting andbroadcasting only the requested one of the different code images. In theevent of a dual master source, code image 130 may also be broadcast, orseparate master source and module function code images may be providedand broadcast. For example, one master source may be mounted at a moduleof a first type and the dual master source at a module of a second type,and another module function may be a processor 105. Where the onebroadcast master source code image comprises a combination of differentcode images, the processors 105 additionally select and implement one ofthe combination of different code images, specifically, the code imagethat is correct for the processor. Where the master source 108 selectsthe desired code image 131-133 in response to a specific request, andbroadcasts the selected code image, each of the requesting processors105 determines whether the broadcast code image is correct for therequesting processor, and the requesting processor selects the broadcastcode image for implementation if the determination is that the codeimage is correct for the processor. A dual master may also request andreceive a code image 130. Alternatively, all processors 105 may use onecommon code image, since they contain many common code elements, buteach processor can sense node type and not execute some portions of thecode.

[0033] Referring to FIGS. 3 and 4, an example of an embedded systemhaving a plurality of modules with processors at nodes of the system,comprises an automated data storage library 10, which stores removabledata storage media 14 in storage shelves 16. An example of an automateddata storage library is the IBM 3494 Tape Library Dataserver. Theexemplary library comprises a base frame 11, may additionally compriseone or more extension frames 12, and may comprise a high availabilityframe 13. The base frame 11 of the library 10 comprises a plurality ofdata storage drives 15 to read and/or write data on the data storagemedia, and has a robot accessor 18 to transport the data storage mediabetween the storage shelves and the data storage drives. The robot 18includes a gripper assembly 20 and may include a bar code scanner orreading system 22 to read identifying information about the data storagemedia 14. The library may also comprise an operator panel 23 or otheruser interface which allows a user to interact with the library.

[0034] The extension frame 12 comprises additional storage shelves, andmay comprise additional data storage drives 15. The high availabilityframe 13 may also comprise additional storage shelves and data storagedrives 15, and comprises a second robot accessor 28, which includes agripper assembly 30 and may include a bar code scanner 32 or otherreading device, and an operator panel 280 or other user interface. Inthe event of a failure or other unavailability of the robot accessor 18,or its gripper 20, etc., the second robot accessor 28 may take over.Additionally, both robot accessors may operate simultaneously indifferent sections of the library to speed storage and retrieval of thedata storage media.

[0035] The library 10 receives commands from one or more host systems40, 41 or 42. The host systems, such as host servers, communicate withthe library, either directly on path 80, or through one or more datastorage drives 15 on paths 81 or 82. The paths 80, 81 or 82 may, forexample, comprise SCSI busses or fibre channel arbitrated loops.

[0036] The embedded system comprises processors at the modules tooperate the library, with a network to couple the modules of theembedded library system. Such a library system may comprise a hundred ormore data storage drive processors and tens of library processors. Thus,the embedded system may comprise communication processor nodes 50, 155,and 250 to receive commands from the hosts and convert the commands tophysical movements of the robot accessors 18, 28. The communicationprocessor nodes 50, 155, and 250 may also provide a communication linkfor operating the data storage drives 15. Work processor nodes 52 and252 are located at the respective robot accessor 18 and 28, and respondto host commands received from the communication processor nodes anddirect the operation of the accessor, providing XY move commands. Thework processor nodes are also coupled to the respective scanners 22, 32.XY processor nodes 55 and 255 are located at an XY system of theaccessor and respond to the XY move commands, operating the XY system toposition the grippers 20, 30 to access the media.

[0037] Also, operator panel processor nodes 59, 259 may be provided atthe respective operator panels 23, 280 for providing an interface forcommunicating between the operator panel and the communication processornodes 50, 155, and 250, the work processor nodes 52, 252, and the XYprocessor nodes 55, 255.

[0038] The network comprises base frame common bus 60, coupling thecommunication processor node 50, the operator panel processor node 59,the work processor node 52, and the XY processor node 55. The extensionframe 12 is coupled by an extension common bus 152 to the base framecommon bus 60. The communication processor node 155 is coupled to theextension common bus 152, and communicates with both data storage drives15 and with hosts, either directly or indirectly, at input 156.Additional extension frames employing identical communication processornodes 155, storage shelves 16, data storage drives 15, and extensionbusses 152, may be provided, and each is coupled to the adjacentextension frame. The high availability frame 13 comprises an extensioncommon bus 200 coupled to the extension common bus 152 of an extensionframe or, if there is no extension frame, to the common bus 60 of thebase frame. The high availability frame extension common bus 200 couplesthe communication processor node 250, which is coupled to the datastorage drives 15 and may receive commands from hosts at input 256, tothe work processor node 252, to the operator panel processor node 259,and to the XY processor node 255.

[0039] Thus, each of the processor nodes is coupled to the common busses60, 152, 200 to form a network.

[0040] The common busses 60, 152, 200 may comprise a wiring network,such as the commercially available “CAN” bus system, which is amulti-drop network, having a standard access protocol and wiringstandards, for example, as defined by CiA, the CAN in AutomationAssociation, Am Weich selgarten 26, D-91058 Erlangen, Germany. Each ofthe extension common busses 152, 200 may comprise a flex cableconnection to the preceding adjacent common bus. Other similar busnetworks may be employed for implementing the present invention.Alternatively, the common busses 60, 152, 200 may comprise a wirelessnetwork system, such as RF or infrared, as is known to those of skill inthe art.

[0041] The processors coupled to the network may comprise a processor ofthe power of a workstation or PC, in the case of the communicationprocessor nodes 50, 155, 250, the work processor nodes 52, 252, the XYprocessor nodes 55, 255, and each of the drives 15, and the code imagefor each module processor may be quite large. Also, each of the operatorpanel processor nodes 59, 259 may comprise a module processor requiringa code image, or may comprise a separate processing system. Further,each of the types of modules conduct different jobs, potentiallyrequiring different code images.

[0042] Referring to FIG. 1, when a module 110 or its processor 105 isrepaired, replaced or upgraded, it must be powered off, the repair, etc.work conducted, and then it is turned on, called a power-on reset. Whenthe processor is powered off, the code image is lost from the RAM 114and must be replaced before the processor, and therefore the module, canbecome operational.

[0043] Referring additionally to FIG. 5, the present invention providescode images to processors of a multi-node processing network withoutrequiring a high capacity NVRAM, PROM or ROM at each processor to storethe code image in the event the module is powered off.

[0044] In step 300, the code image, or code images 131-133, or codeimage 130 in the case a dual master, of FIG. 2, are loaded to the mastersource 108 and, in step 301, are stored in the non-volatile memory 126of the master source. In FIGS. 1-5, the master source 108 may compriseany of the processors coupled to the base frame common bus 60, and may,for example, comprise the work processor node 52. The code images131-133 each comprises the current updated code image for thecorresponding processors 105. The code images are thus ready to bebroadcast when requested by one or more processors.

[0045] As discussed above, a processor 105 may be rebooted, depicted asstep 305, such that the boot program stored in the non-volatile memory113 becomes operational, and is provided to the processing unit 115 ofthe processor 105 in step 307. In accordance with the present invention,the non-volatile memory 113 is of relatively small capacity, and onlycontains the boot code. The boot code may first conduct at least a basicsystem test which tests the components of the module, in step 309, andthen places the processor 105 in the minimally operational state in step310. The processor may previously have been powered off and subsequentlypowered on and is, at that time, absent a code image required to becomefully operational. Then, in step 311, the processor employs the bootprogram to request the code image from the network 106.

[0046] The master source receives the first code image request in step320. In accordance with the present invention, upon receiving a firstcode image request from a processor 105 at the master interface 120, themaster source 108, in step 321, waits a predetermined time period. Thepredetermined time period of step 321 is sufficient to allow anyadditional processor 105 that might be powered on at about the sametime, to reach the minimally operational state, and provide a code imagerequest, received at step 322. The predetermined time may be fixed orvariable, and may be timed from the first received request, oralternatively from the most recently received request. Thus, as anexample, if several data storage drives 15 were replaced or upgraded atthe same time, they could be powered on at about the same time, or aframe could be powered on and the master source 108 waits for allprocessors to reach the minimally operational state so that the codeimage will only be broadcast once.

[0047] In the event the predetermined time period is timed from the mostrecently received request, step 324 determines whether a new time periodis required. If so, the previous time period is renewed in step 321, or,alternatively, a new predetermined time period is established in step321, in effect extending the predetermined time period. As an example,the new predetermined time period may be shorter than the previous timeperiod by a fixed or variable amount. If no new request is received, orif the timing is only from the first received request, the processproceeds to step 323. Step 323 determines whether the time period hasexpired, and, if not, continues the wait step 321.

[0048] Referring additionally to FIG. 2, as discussed above, theprocessors 105 of a multi-node network of embedded systems 100 areassociated with modules that perform various functions in the system,and may employ correspondingly different code images. As the result, themaster source 108 of the present invention may store multiple codeimages 131-133 for operating the various processors 105.

[0049] The master source 108 may distribute the code images 131-133either by providing and broadcasting one code image, which may be all ofthe code images 131-133, for any code image request, or may optionallyrespond to a specific request for a specific code image. Thus, as anoption, upon completion of the predetermined time period, as determinedby step 323, the master source 108, in step 330, determines whether thecode image request received in step 320, and any code image requestreceived in step 322, is a specific request for one of the multiple codeimages 131-133. For example, an XY processor node 55, 255 may request aspecific code image by accompanying the request with a specificidentifier, such as an encoded number for the type of module.

[0050] If step 330 indicates that a specific code image has beenrequested, the master source selects that code image from non-volatilememory 126 in step 331.

[0051] The master source 108 then, in step 335, distributes the codeimages 131-133 either by 1) providing and broadcasting one, possiblycombined, code image for the code image request, or by 2) broadcastingonly the requested one of the different code images selected in step331. A redundant master source may also request a code image 130, whichis selected by master source 108 in step 331 and supplied in step 335.

[0052] The broadcast code images are supplied at the interface 112 ofeach of the processors 105 in step 336. In accordance with an aspect ofthe present invention, if the processor is in the fully operationalstate and running, that operation will not be interrupted or disturbed,and, in step 337, the code image is ignored and not received. If theprocessor is in the minimally operational state, the broadcast codeimage(s) are received in step 338. The code for making the determinationof step 336 and conducting step 337 or step 338 may be part of the bootprogram of non-volatile memory 113, or part of the operational code ofRAM 114.

[0053] In an optional situation where, either the master processor 108is capable of broadcasting only a single code image in step 331 at onetime, the specific code image of a first specific request for a codeimage of step 311, may be broadcast even though other processorsrequested different code images, or a single code image comprising acombination of code images 131-133 is broadcast, then, each of theprocessors 105 must determine whether it is receiving a correct codeimage. If that option is available, step 340 determines that theprocessor must select the correct code image. The determination whetherthe code image is correct, or the correct code image is included in thebroadcast, is made in step 343. If the code is correct, it is selectedin step 345, and, if not, it is ignored in step 346 and the processcycles back to step 338 to receive the next broadcast.

[0054] Thus, where the one broadcast code image comprises a combinationof different code images, the requesting processors 105 additionallyselect, in step 343, the code image that is correct for the processor.Where the broadcast code image is one of the different code images131-133 selected by the master source 108 in response to a specificrequest, each of the requesting processors 105 determines, in step 343,whether the broadcast code image is correct for the processor, andselects the broadcast code image for implementation if the determinationis that the code image is correct for the processor.

[0055] Whether selected by step 345, or if no selection is required fromstep 340, the code image is stored in RAM 114 and implemented in step350. As is known to those of skill in the art, the code image mayoverwrite any data previously in RAM 114. As an example, the bootprogram of non-volatile memory 113 may begin execution of the code imagein step 350. The code for making the determinations of steps 340 and343, the selection and ignoring steps 345 and 346, and the storage ofthe code image are also part of the boot program of non-volatile memory113.

[0056] In the example of an embedded system of FIGS. 3 and 4, the mastersource may comprise work processor 52, and the processors which mayrequest code images may comprise the processors associated with the datastorage drives 15, or may comprise the communication processor nodes 50,155, 250, the work processor node 252, the XY processor nodes 55, 255,or the operator panel processor nodes 59, 259. Also, a backup mastersource may be provided, for example, at the work processor node 252.

[0057] As the result, the present invention, without requiring a highcapacity NVRAM, PROM or ROM at each processor for storing the entirecode image during a power off situation, provides the correct codeimages to each of the requesting processors of a multi-node processingnetwork.

[0058] In embedded systems which, for example, require continuousoperation, such as an automated data storage library, the modules arepreferably redundant, or “hot-swappable”, so that they may be repaired,replaced or upgraded as needed without requiring turning off the entiresystem. In the example of FIGS. 3 and 4, the robot accessors may beredundant, such that work processor 252 contains the same code image aswork processor 52, and may also comprise a backup master source, ifneeded. Also redundant are the XY processor nodes 55, 255. The operatorpanel processor nodes 59, 259, and the communication processor nodes 50,155, 250 may also be redundant, as may the set of data storage drives 15in each of the frames 11-13.

[0059] In a non-redundant system, a code image for the processors may beupdated by a reboot of the processors that are on the network inaccordance with the present invention, such that the processors willrequest the code image, and the master source will supply the updatedcode image.

[0060] Referring additionally to FIG. 6, in another aspect of thepresent invention, in which the processors are provided for modules of aredundant system, such as an automated data storage library 10, havingat least two sets of modules, the master source receives and stores theupdate code image(s) in step 360. A first set of the redundant modulesis taken offline and rebooted in step 362, while at least a second setcontinues in operation in step 363. As an example, work processor 52 maycomprise the master source, and the modules of the high availabilityframe 13 may be taken offline and rebooted, comprising processors 252,255, 250, 259, and the processors associated with the data storagedrives 15 in the frame. In step 363, the modules of the base frame 11and extension frame 12 may continue operation, comprising processors 52,55, 50, 155, 59, and the processors associated with the data storagedrives 15 in the frames. In one embodiment, the master source 52 maysend messages to cause the processor to reboot and run the boot code asthough the module is being reset after a power-on or is beingreinitialized as discussed above, and, in another embodiment, themodules are rebooted by an operator or an external command.

[0061] In step 366, the boot code is executed, such that the processorsof the modules reach the minimally operational state and request thecode image from the network. The master source 52, upon receiving a codeimage request in step 370, waits a predetermined time period, thepredetermined time period allowing any additional processor of the firstset of modules to reach the minimally operational state and request codeimages. The master source 52, upon completion of the predetermined timeperiod, in step 373, broadcasts the code image(s) on the network, suchthat the processors requesting the code image(s) each receive the codeimage(s) in step 374, store the correct code image, and execute the codeimage and become fully operational.

[0062] Then, in step 380, a remaining second set of redundant modules isrebooted, as above. For example, the modules of the base frame 11 andextension frame 12 may be taken offline and rebooted, comprisingprocessors 55, 50, 155, 59, and the processors associated with the datastorage drives 15 in the frames. Master source 52, or 252, if redundant,remains operational, as do the modules that became operational in step374. Upon being rebooted, the processors 105 of the remaining set ofredundant modules, in step 382, reach the minimally operational stateand request the code image from the network. The master source 52, instep 385, upon receiving a code image request, waits a predeterminedtime period, the predetermined time period allowing any additionalprocessor of the second set to reach the minimally operational state andrequest a code image. In step 388, upon completion of the predeterminedtime period, the master source 52 broadcasts the code image(s) on thenetwork. In step 390, the second redundant set of processors requestingthe code image(s) each receive the code image(s), store the correct codeimage, and execute the code image and become fully operational. Steps380-390 may be repeated for further sets of redundant modules.

[0063] In this manner, the code images for the entire redundant systemhave been updated without requiring that the entire system beinterrupted.

[0064] As the result, the present invention eliminates the need tomaintain and update code levels on individual processors.

[0065] While the preferred embodiments of the present invention havebeen illustrated in detail, it should be apparent that modifications andadaptations to those embodiments may occur to one skilled in the artwithout departing from the scope of the present invention as set forthin the following claims.

I claim:
 1. A multi-node network of processors, comprising: a network; aplurality of processors coupled in said network, said processors havinga minimally operational state, and having a fully operational stateemploying a code image, said processors, when in said minimallyoperational state, requesting said code image from said network; and amaster source coupled in said network, said master source having atleast said code image for broadcasting said code image on said network,said master source, upon receiving said code image request, waiting apredetermined time period, said predetermined time period allowing anyadditional said processor to reach said minimally operational state,and, upon completion of said predetermined time period, broadcastingsaid code image on said network.
 2. The multi-node network of processorsof claim 1, wherein said processors, additionally, upon said broadcastof said code image, receive and implement said code image only if saidprocessor is in said minimally operational state.
 3. The multi-nodenetwork of processors of claim 1, wherein said processors additionallyeach comprises a non-volatile memory for storing said minimallyoperational state code.
 4. The multi-node network of processors of claim3, wherein said minimally operational state code comprises a bootprogram which becomes operational upon reboot of said processor.
 5. Themulti-node network of processors of claim 3, wherein said code for saidminimally operational state is additionally sufficient to conduct atleast a basic system test and provide said code image request.
 6. Themulti-node network of processors of claim 1, wherein said processorsadditionally comprise a RAM for, upon receiving said code image, storingsaid code image.
 7. The multi-node network of processors of claim 1,wherein said master source provides one said code image for any saidcode image request.
 8. The multi-node network of processors of claim 7,wherein ones of said processors implement different said code images,wherein said one master source code image comprises a combination ofsaid different code images, and wherein said processors additionallyselect and implement one of said combination of different code images.9. The multi-node network of processors of claim 1, wherein said mastersource comprises a plurality of different said code images, wherein saidprocessor requesting said code image requests one of said different codeimages, wherein said master source broadcasts said requested one of saiddifferent code images, and wherein said processors additionallydetermine whether said broadcast code image is correct for saidprocessor, and select said broadcast code image for implementation ifsaid determination determines that said code image is correct for saidprocessor.
 10. A method for providing a code image for processing nodesof a multi-node network of processors, comprising the steps of: at leastone said processor, comprising a node of said network, in a minimallyoperational state, requesting said code image from said network; amaster source, upon receiving said code image request, waiting apredetermined time period, said predetermined time period allowing anyadditional said processor to reach said minimally operational state; andsaid master source, upon completion of said predetermined time period,broadcasting said code image on said network.
 11. The method of claim10, additionally comprising the step of, said processors, upon saidbroadcast of said code image, receiving and implementing said code imageonly if said processor is in said minimally operational state.
 12. Themethod of claim 10, wherein said code of said minimally operationalstate is stored in a non-volatile memory.
 13. The method of claim 12,wherein said minimally operational state additionally comprisesoperation of a boot program which becomes operational upon reboot ofsaid processor.
 14. The method of claim 12, wherein said minimallyoperational state of said processor comprises having said non-volatilecode additionally sufficient to conduct at least a basic system test andprovide said code image request.
 15. The method of claim 10,additionally comprising the step of, said processor, upon receiving saidcode image, storing said code image in RAM.
 16. The method of claim 10,wherein one said code image is provided by said master source for anysaid code image request.
 17. The method of claim 16, wherein ones ofsaid processors implement different said code images, wherein said onecode image provided by said master source comprises a combination ofsaid different code images, and wherein said method additionallycomprises the step of, said processors selecting and implementing one ofsaid combination of different code images.
 18. The method of claim 10,wherein said master source comprises a plurality of different said codeimages, wherein said step of said processor requesting said code imagecomprises requesting one of said different code images, wherein saidstep of said master source broadcasting said code image comprisesbroadcasting said requested one of said different code images, andwherein said method additionally comprises the step of, a receiving saidprocessor determining whether said broadcast code image is correct forsaid processor, and selecting said code image for implementation if saiddetermination step determines that said code image is correct for saidprocessor.
 19. For a multi-node network of processors, said networkhaving a plurality of processors coupled in said network, saidprocessors having a minimally operational state, and having a fullyoperational state employing a code image, said processors, when in saidminimally operational state, requesting said code image from saidnetwork, a master source comprising: a master interface coupled in saidnetwork; a memory storing at least said code image for said processorsof said network; and a master processor coupled to said memory and tosaid master interface, upon receiving said code image request at saidmaster interface, waiting a predetermined time period, saidpredetermined time period allowing any additional said processor toreach said minimally operational state, and, upon completion of saidpredetermined time period, broadcasting said code image stored in saidmemory, via said master interface, on said network.
 20. The mastersource for said multi-node network of processors of claim 19, whereinsaid master processor provides one said code image for any said codeimage request.
 21. The master source for said multi-node network ofprocessors of claim 20, wherein ones of said processors implementdifferent said code images, and wherein said one master source codeimage stored in said memory and broadcast by said master processorcomprises a combination of said different code images, such that saidprocessors additionally select and implement one of said combination ofdifferent code images.
 22. The master source of said multi-node networkof processors of claim 19, comprising a plurality of different said codeimages, wherein said processor requesting said code image requests oneof said different code images, wherein said master source masterprocessor selects from said memory and broadcasts said requested one ofsaid different code images, such that a receiving said processoradditionally determines whether said broadcast code image is correct forsaid processor, and selects said broadcast code image for implementationif said determination determines that said code image is correct forsaid processor.
 23. For a multi-node network of processors, said networkhaving a master source coupled in said network, said master sourcehaving a code image for broadcasting on said network, said mastersource, upon receiving said code image request, waiting a predeterminedtime period, said predetermined time period allowing any additionalprocessor to reach said minimally operational state, and, uponcompletion of said predetermined time period, broadcasting saidrequested code image on said network, a processor comprising: aprocessor interface coupling said processor in said network; anon-volatile memory for storing code providing a minimally operationalstate of said processor; a processor memory storing a code imageproviding a fully operational state of said processor; and a processingunit coupled to said non-volatile memory, said processor memory and saidprocessor interface, when in said minimally operational state providedby said non-volatile memory, requesting said code image from saidnetwork, via said processor interface.
 24. The processor for amulti-node network of processors of claim 23, wherein said processingunit, additionally, upon said broadcast of said code image by saidmaster source, receives and stores said code image in said processormemory, and implements said code image, only if said processing unit isin said minimally operational state.
 25. The processor for a multi-nodenetwork of processors of claim 23, wherein said minimally operationalstate code stored in said non-volatile memory comprises a boot programwhich is provided to said processing unit and becomes operational uponreboot of said processor.
 26. The processor for a multi-node network ofprocessors of claim 25, wherein said boot program stored in saidnon-volatile memory storing for said minimally operational state isadditionally sufficient to conduct at least a basic system test andprovide said code image request.
 27. The processor for a multi-nodenetwork of processors of claim 23, wherein said processor memorycomprises a RAM for storing said code image.
 28. The processor for amulti-node network of processors of claim 23, wherein said master sourceprovides one said code image for any said code image request, said onecode image comprising a combination of said different code images, andwherein ones of said processors implement different said one codeimages, said processing unit additionally selecting, storing andimplementing one of said combination of different code images.
 29. Theprocessor of a multi-node network of processors of claim 23, whereinsaid master source comprises a plurality of different said code images,said master source broadcasting said requested one of said differentcode images, and wherein said processor requesting said code imagerequests one of said different code images, said processor additionallydetermines whether said broadcast code image is correct for saidprocessor, and selects said broadcast code image for implementation ifsaid determination determines that said code image is correct for saidprocessor.
 30. A computer program product usable with a programmablecomputer having computer readable program code embodied therein, saidprogrammable computer comprising a master source coupled in a network,said network having a plurality of processors coupled in said network,said processors having a minimally operational state, and having a fullyoperational state employing a code image, said processors, when in saidminimally operational state, requesting said code image from saidnetwork, said computer program product comprising: computer readableprogram code which causes said master source programmable computer tostore at least said code image; computer readable program code whichcauses said master source programmable computer to, upon receiving saidcode image request, wait a predetermined time period, said predeterminedtime period allowing any additional said processor to reach saidminimally operational state; and computer readable program code whichcauses said master source programmable computer to, upon completion ofsaid time period, broadcast said stored code image on said network. 31.The computer program product of claim 30, wherein ones of saidprocessors implement different said code images; wherein said storedcode image comprises a combination of said different code images; andwherein said computer readable program code which causes said mastersource computer processor to broadcast said stored code image, causessaid master source computer processor to broadcast said code imagecombination of said different code images as one code image, such thatsaid processors additionally select and implement one of saidcombination of different code images.
 32. The computer program productof claim 30, wherein said stored code image comprises a plurality ofdifferent said code images, wherein said processor requesting said codeimage requests one of said different code images, and wherein saidcomputer readable program code which causes said master source computerprocessor to broadcast said stored code image, causes said master sourcecomputer processor to broadcast said requested one of said differentcode images, such that said processors additionally determine whethersaid broadcast code image is correct for said processor, and select saidbroadcast code image for implementation if said determination determinesthat said code image is correct for said processor.
 33. A method forupdating code images for processors of modules of a redundant system,said redundant system comprising at least two sets of redundant saidmodules, said processors comprising nodes of a multi-node network ofprocessors, said processors having a minimally operational state andrequiring a code image to become fully operational, said processors,when in said minimally operational state, request said code image fromsaid network, said method comprising the steps of: providing a reboot ofat least one said set of redundant modules, such that said processors ofsaid modules reach said minimally operational state and request saidcode image from said network; a master source, upon receiving said codeimage request, waiting a predetermined time period, said predeterminedtime period allowing any additional said processor to reach saidminimally operational state; said master source, upon completion of saidpredetermined time period, broadcasting said code image on said network,such that said processors requesting said code image become fullyoperational; providing a reboot of a remaining at least one said set ofredundant modules, such that said processors of said modules reach saidminimally operational state and request said code image from saidnetwork; said master source, upon receiving said code image request,waiting a predetermined time period, said predetermined time periodallowing any additional said processor to reach said minimallyoperational state; and said master source, upon completion of saidpredetermined time period, broadcasting said code image on said network,such that said processors of said remaining set of redundant modulesrequesting said code image become fully operational.
 34. A computerprogram product usable with a programmable computer having computerreadable program code embodied therein, said programmable computercomprising a master source coupled in a network, for updating codeimages for processors of modules of a redundant system, said redundantsystem comprising at least two sets of redundant said modules, saidprocessors comprising nodes of a multi-node network of processors, saidprocessors having a minimally operational state and requiring a codeimage to become fully operational, said processors, when in saidminimally operational state, request said code image from said networksaid computer program product comprising: computer readable program codewhich causes said master source programmable computer to store at leastan update of said code image; computer readable program code whichcauses said master source programmable computer to provide a reboot ofat least one said set of redundant modules, such that said processors ofsaid modules reach said minimally operational state and request saidcode image from said network; computer readable program code whichcauses said master source programmable computer to, upon receiving saidcode image request, wait a predetermined time period, said predeterminedtime period allowing any additional said processor to reach saidminimally operational state; computer readable program code which causessaid master source programmable computer to, upon completion of saidpredetermined time period, broadcast said code image on said network,such that said processors requesting said code image become fullyoperational; computer readable program code which causes said mastersource programmable computer to provide a reboot of a remaining at leastone said set of redundant modules, such that said processors of saidmodules reach said minimally operational state and request said codeimage from said network; computer readable program code which causessaid master source programmable computer to, upon receiving said codeimage request, wait a predetermined time period, said predetermined timeperiod allowing any additional said processor to reach said minimallyoperational state; and computer readable program code which causes saidmaster source programmable computer to, upon completion of saidpredetermined time period, broadcast said code image on said network,such that said processors of said remaining set of redundant modulesrequesting said code image become fully operational.
 35. A multi-nodenetwork of processors, comprising: a network; a first set of redundantprocessors coupled in said network, said processors having a minimallyoperational state, and having a fully operational state employing a codeimage, said processors, when in said minimally operational state,requesting said code image from said network; a second set of redundantprocessors coupled in said network, said processors having a minimallyoperational state, and having a fully operational state employing a codeimage, said processors, when in said minimally operational state,requesting said code image from said network; and a master sourcecoupled in said network, said master source having at least said codeimage for broadcasting said code image on said network, said first setof redundant processors rebooted to said minimally operational state,said master source, upon receiving said code image request from one ofsaid first set of redundant processors, waiting a predetermined timeperiod, said predetermined time period allowing any additional saidprocessor to reach said minimally operational state, and, uponcompletion of said predetermined time period, broadcasting said codeimage on said network, whereby said first set of redundant processorsbecome fully operational; whereupon said second set of redundantprocessors are subsequently rebooted to said minimally operationalstate, said master source, upon receiving said code image request fromone of said second set of redundant processors, waiting a predeterminedtime period, said predetermined time period allowing any additional saidprocessor to reach said minimally operational state, and, uponcompletion of said predetermined time period, broadcasting said codeimage on said network, whereby said second set of redundant processorsbecome fully operational.